Search Results: "philipp"

11 September 2020

Louis-Philippe V ronneau: Hire me!

I'm happy to announce I handed out my Master's Thesis last Monday. I'm not publishing the final copy just yet1, as it still needs to go through the approval committee. If everything goes well, I should have my Master of Economics diploma before Christmas! It sure hasn't been easy, and although I regret nothing, I'm also happy to be done with university. Looking for a job What an odd time to be looking for a job, right? Turns out for the first time in 12 years, I don't have an employer. It's oddly freeing, but also a little scary. I'm certainly not bitter about it though and it's nice to have some time on my hands to work on various projects and read things other than academic papers. Look out for my next blog posts on using the NeTV2 as an OSHW HDMI capture card, on hacking at security tokens and much more! I'm not looking for anything long term (I'm hoping to teach Economics again next Winter), but for the next few months, my calendar is wide open. For the last 6 years, I worked as Linux system administrator, mostly using a LAMP stack in conjunction with Puppet, Shell and Python. Although I'm most comfortable with Puppet, I also have decent experience with Ansible, thanks to my work in the DebConf Videoteam. I'm not the most seasoned Debian Developer, but I have some experience packaging Python applications and libraries. Although I'm no expert at it, lately I've also been working on Clojure packages, as I'm trying to get Puppet 6 in Debian in time for the Bullseye freeze. At the rate it's going though, I doubt we're going to make it... If your company depends on Puppet and cares about having a version in Debian 11 that is maintained (Puppet 5 is EOL in November 2020), I'm your guy! Oh, and I guess I'm a soon-to-be Master of Economics specialising in Free and Open Source Software business models and incentives theory. Not sure I'll ever get paid putting that in application, but hey, who knows. If any of that resonates with you, contact me and let's have a chat! I promise I don't bite :)
  1. The title of the thesis is What are the incentive structures of Free Software? An economic analysis of Free Software's specific development model. Once the final copy is approved, I'll be sure to write a longer blog post about my findings here.

23 August 2020

Philipp Kern: Self-service buildd givebacks now use Salsa auth

As client certificates are on the way out and Debian's SSO solution is effectively not maintained any longer, I switched self-service buildd givebacks over to Salsa authentication. It lives again at https://buildd.debian.org/auth/giveback.cgi. For authorization you still need to be in the "debian" group for now, i.e. be a regular Debian member.For convenience the package status web interface now features an additional column "Actions" with generated "giveback" links.Please remember to file bugs if you give builds back because of flakiness of the package rather than the infrastructure and resist the temptation to use this excessively to let your package migrate. We do not want to end up with packages that require multiple givebacks to actually build in stable, as that would hold up both security and stable updates needlessly and complicate development.

16 July 2020

Louis-Philippe V ronneau: DebConf Videoteam Sprint Report -- DebConf20@Home

DebConf20 starts in about 5 weeks, and as always, the DebConf Videoteam is working hard to make sure it'll be a success. As such, we held a sprint from July 9th to 13th to work on our new infrastructure. A remote sprint certainly ain't as fun as an in-person one, but we nonetheless managed to enjoy ourselves. Many thanks to those who participated, namely: We also wish to extend our thanks to Thomas Goirand and Infomaniak for providing us with virtual machines to experiment on and host the video infrastructure for DebConf20. Advice for presenters For DebConf20, we strongly encourage presenters to record their talks in advance and send us the resulting video. We understand this is more work, but we think it'll make for a more agreeable conference for everyone. Video conferencing is still pretty wonky and there is nothing worse than a talk ruined by a flaky internet connection or hardware failures. As such, if you are giving a talk at DebConf this year, we are asking you to read and follow our guide on how to record your presentation. Fear not: we are not getting rid of the Q&A period at the end of talks. Attendees will ask their questions either on IRC or on a collaborative pad and the Talkmeister will relay them to the speaker once the pre-recorded video has finished playing. New infrastructure, who dis? Organising a virtual DebConf implies migrating from our battle-tested on-premise workflow to a completely new remote one. One of the major changes this means for us is the addition of Jitsi Meet to our infrastructure. We normally have 3 different video sources in a room: two cameras and a slides grabber. With the new online workflow, directors will be able to play pre-recorded videos as a source, will get a feed from a Jitsi room and will see the audience questions as a third source. This might seem simple at first, but is in fact a very major change to our workflow and required a lot of work to implement. 
               == On-premise ==                                          == Online ==
                                                      
              Camera 1                                                 Jitsi
                                                                          
                 v                 ---> Frontend                         v                 ---> Frontend
                                                                                            
    Slides -> Voctomix -> Backend -+--> Frontend         Questions -> Voctomix -> Backend -+--> Frontend
                                                                                            
                 ^                 ---> Frontend                         ^                 ---> Frontend
                                                                          
              Camera 2                                           Pre-recorded video
In our tests, playing back pre-recorded videos to voctomix worked well, but was sometimes unreliable due to inconsistent encoding settings. Presenters will thus upload their pre-recorded talks to SReview so we can make sure there aren't any obvious errors. Videos will then be re-encoded to ensure a consistent encoding and to normalise audio levels. This process will also let us stitch the Q&As at the end of the pre-recorded videos more easily prior to publication. Reducing the stream latency One of the pitfalls of the streaming infrastructure we have been using since 2016 is high video latency. In a worst case scenario, remote attendees could get up to 45 seconds of latency, making participation in events like BoFs arduous. In preparation for DebConf20, we added a new way to stream our talks: RTMP. Attendees will thus have the option of using either an HLS stream with higher latency or an RTMP stream with lower latency. Here is a comparative table that can help you decide between the two protocols:
HLS RTMP
Pros
  • Can be watched from a browser
  • Auto-selects a stream encoding
  • Single URL to remember
  • Lower latency (~5s)
Cons
  • Higher latency (up to 45s)
  • Requires a dedicated video player (VLC, mpv)
  • Specific URLs for each encoding setting
Live mixing from home with VoctoWeb Since DebConf16, we have been using voctomix, a live video mixer developed by the CCC VOC. voctomix is conveniently divided in two: voctocore is the backend server while voctogui is a GTK+ UI frontend directors can use to live-mix. Although voctogui can connect to a remote server, it was primarily designed to run either on the same machine as voctocore or on the same LAN. Trying to use voctogui from a machine at home to connect to a voctocore running in a datacenter proved unreliable, especially for high-latency and low bandwidth connections. Inspired by the setup FOSDEM uses, we instead decided to go with a web frontend for voctocore. We initially used FOSDEM's code as a proof of concept, but quickly reimplemented it in Python, a language we are more familiar with as a team. Compared to the FOSDEM PHP implementation, voctoweb implements A / B source selection (akin to voctogui) as well as audio control, two very useful features. In the following screen captures, you can see the old PHP UI on the left and the new shiny Python one on the right. The old PHP voctowebThe new Python3 voctoweb Voctoweb is still under development and is likely to change quite a bit until DebConf20. Still, the current version seems to works well enough to be used in production if you ever need to. Python GeoIP redirector We run multiple geographically-distributed streaming frontend servers to minimize the load on our streaming backend and to reduce overall latency. Although users can connect to the frontends directly, we typically point them to live.debconf.org and redirect connections to the nearest server. Sadly, 6 months ago MaxMind decided to change the licence on their GeoLite2 database and left us scrambling. To fix this annoying issue, Stefano Rivera wrote a Python program that uses the new database and reworked our ansible frontend server role. Since the new database cannot be redistributed freely, you'll have to get a (free) license key from MaxMind if you to use this role. Ansible & CI improvements Infrastructure as code is a living process and needs constant care to fix bugs, follow changes in DSL and to implement new features. All that to say a large part of the sprint was spent making our ansible roles and continuous integration setup more reliable, less buggy and more featureful. All in all, we merged 26 separate ansible-related merge request during the sprint! As always, if you are good with ansible and wish to help, we accept merge requests on our ansible repository :)

11 June 2020

Louis-Philippe V ronneau: How to capture a remote IRC session live

DebConf20 will be held online this year and I've started doing some work for the DebConf videoteam to prepare what's to come. One thing I want us to do is capture a live IRC session and use it as a video input in Voctomix, the live video mixer we use. This way, at the end of a talk we could show both the attendees asking questions on IRC and the presenter replying to them side-by-side. A mockup of a side-by-side voctogui window with someone on the left and a terminal running weechat on the right Capturing a live video of an IRC client on a remote headless server is somewhat more complicated than you might think; as far as I know, neither ffmpeg nor gstreamer support recording a live ssh pseudoterminal1. Worse, neither weechat nor irssi run on X: they use ncurses... Although you can capture an X11 window with ffmpeg -f x11grab, I wasn't able to get them to run with Xvfb. Capturing the framebuffer One thing I dislike with this method is the framebuffer isn't always easy to access on remote machines. If you don't have a serial connection, you can try using a VNC server that can. I did my tests in a VM on an KVM hypervisor and used virt-manager to access the framebuffer. I had a hard time setting the framebuffer resolution to a 16:9 aspect ratio. The winning combination ended up passing the nomodeset kernel parameter at boot and setting up these parameters in /etc/default/grub2: 
GRUB_GFXMODE=1280x720
GRUB_GFXPAYLOAD_LINUX=keep
To make the text more readable, this is the /etc/default/console-setup file that seemed to make the most sense: 
# CONFIGURATION FILE FOR SETUPCON
# Consult the console-setup(5) manual page.
ACTIVE_CONSOLES="/dev/tty[1-6]"
CHARMAP="UTF-8"
CODESET="Lat15"
FONTFACE="TerminusBold"
FONTSIZE="12x24"
Once that is done, the only thing left is to run the IRC client and launch ffmpeg. The magic command to record the framebuffer seems to be something like: 
ffmpeg -f fbdev -framerate 60 -i /dev/fb0 -c:v libvpx -crf 10 -b:v 1M -auto-alt-ref 0 output.webm
Here is what I ended up with:
  1. We need something similarly flexible and featureful that can output to a TCP socket.
  2. Don't forget to run update-grub before rebooting!

8 April 2020

Louis-Philippe V ronneau: Using Jitsi Meet with Puppet for self-hosted video conferencing

Here's a blog post I wrote for the puppet.com blog. Many thanks to Ben Ford and all their team!. With everything that is currently happening around the world, many of us IT folks have had to solve complex problems in a very short amount of time. Pretty quickly at work, I was tasked with finding a way to make virtual meetings easy, private and secure. Whereas many would have turned to a SaaS offering, we decided to use Jitsi Meet, a modern and fully on-premise FOSS videoconferencing solution. Jitsi works on all platforms by running in a browser and comes with nifty Android and iOS applications. We've been using our instance quite a bit, and so far everyone from technical to non-technical users have been pretty happy with it. Jitsi Meet is powered by WebRTC and can be broken into multiple parts across multiple machines if needed. In addition to the webserver running the Jitsi Meet JavaScript code, the base configuration uses the Videobridge to manage users' video feeds, Jicofo as a conference focus to manage media sessions and the Prosody XMPP server to tie it all together. Here's a network diagram I took from their documentation to show how those applications interact: A network diagram that shows how the different bits of jitsi meet work together Getting started with the Jitsi Puppet module First of all, you'll need a valid domain name and a server with decent bandwidth. Jitsi has published a performance evaluation of the Videobridge to help you spec your instance appropriately. You will also need to open TCP ports 443, 4443 and UDP port 10000 in your firewall. The puppetlabs/firewall module could come in handy here. Once that is done, you can use the smash/jitsimeet Puppet module on a Debian 10 (Buster) server to spin up an instance. A basic configuration would look like this: 
  class   'jitsimeet':
    fqdn                 => 'jitsi.example.com',
    repo_key             => puppet:///files/apt/jitsimeet.gpg,
    manage_certs         => true,
    jitsi_vhost_ssl_key  => '/etc/letsencrypt/live/jitsi.example.com/privkey.pem'
    jitsi_vhost_ssl_cert => '/etc/letsencrypt/live/jitsi.example.com/cert.pem'
    auth_vhost_ssl_key   => '/etc/letsencrypt/live/auth.jitsi.example.com/privkey.pem'
    auth_vhost_ssl_cert  => '/etc/letsencrypt/live/auth.jitsi.example.com/cert.pem'
    jvb_secret           => 'mysupersecretstring',
    focus_secret         => 'anothersupersecretstring',
    focus_user_password  => 'yetanothersecret',
    meet_custom_options  =>  
      'enableWelcomePage'         => true,
      'disableThirdPartyRequests' => true,
     ;
The jitsimeet module is still pretty young: it clearly isn't perfect and some external help would be very appreciated. If you have some time, here are a few things that would be nice to work on: If you use this module to manage your Jitsi Meet instance, please send patches and bug reports our way! Learn more

31 March 2020

Chris Lamb: Free software activities in March 2020

Here is my monthly update covering what I have been doing in the free software world during March 2020 (previous month): In addition, I did even more hacking on the Lintian static analysis tool for Debian packages, including:
Reproducible builds One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised. The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter. This month, I: In our tooling, I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138 to Debian: The Reproducible Builds project also operates a fully-featured and comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, I reworked the web-based package rescheduling tool to:
Debian LTS This month I have worked 18 hours on Debian Long Term Support (LTS) and 8 hours on its sister Extended LTS project. You can find out more about the Debian LTS project via the following video:
Debian Uploads For the Debian Privacy Maintainers team I requested that the pyptlib package be removed from the archive (#953429) as well as uploading onionbalance (0.1.8-6) to fix test failures under Pytest 3.x (#953535) and a new upstream release of nautilus-wipe. Finally, I sponsored an upload of bilibop (0.6.1) on behalf of Yann Amar.

30 March 2020

Louis-Philippe V ronneau: Using Zoom's web client on Linux

TL;DR: The zoom meeting link you have probably look like this: 
https://zoom.us/j/123456789
To use the web client, use this instead: 
https://zoom.us/wc/join/123456789
Avant-propos Like too many institutions, the school where I teach chose to partner up with Zoom. I wasn't expecting anything else, as my school's IT department is a Windows shop. Well, I guess I'm still a little disappointed. Although I had vaguely heard of Zoom before, I had never thought I'd be forced to use it. Lucky for me, my employer decided not to force us to use it. To finish the semester, I plan to record myself and talk with my students on a Jitsi Meet instance. I will still have to attend meetings on Zoom though. I'm well aware of Zoom's bad privacy record and I will not install their desktop application. Zoom does offer a web client. Sadly, on Linux you need to jump through hoops to be able to use it. Using Zoom's web client on Linux Zoom's web client apparently works better on Chrome, so I decided to use Chromium. Without already having the desktop client installed on your machine, the standard procedure to use the web client would be:
  1. Open the link to the meeting in Chromium
  2. Click on the "download & run Zoom" link showed on the page
  3. Click on the "join from your browser" link that then shows up
Sadly, that's not what happens on Linux. When you click on the "download & run Zoom" link, it brings you to a page with instructions on how to install the desktop client on Linux. You can thwart that stupid behavior by changing your browser's user agent to make it look like you are using Windows. This is the UA string I've been using: 
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36
With that, when you click on the "download & run Zoom" link, it will try to download a .exe file. Cancel the download and you should now see the infamous "join from your browser" link. Upon closer inspection, it seem you can get to the web client by changing the meeting's URL. The zoom meeting link you have probably look like this: 
https://zoom.us/j/123456789
To use the web client, use this instead: 
https://zoom.us/wc/join/123456789
Jitsi Meet Puppet Module I've been playing around with Jitsi Meet quite a bit recently and I've written a Puppet module to install and configure an instance! The module certainly isn't perfect, but should wield a working Jitsi instance. If you already have a Puppet setup, please give it a go! I'm looking forward receiving feedback (and patches) to improve it.

20 March 2020

Louis-Philippe V ronneau: Today (March 20th 2020) is the day to buy music on Bandcamp

Hey folks, This is a quick blog post to tell you Bandcamp is waiving all their fees on March 20th 2020 (PST). Spread the word, as every penny spent on the platform that day will go back to the artists. COVID-19 is throwing us all a mean curveball and artists have it especially rough, particularly those who were in the middle of tours and had to cancel them. If you like Metal, Angry Metal Guy posted a very nice list of artists you might know and want to help out. If you are lucky enough to have a little coin around, now is the time to show support for the artists you like. Buy an album you liked and copied from a friend or get some merch to wear to your next virtual beer night with your (remote) friends! Stay safe and don't forget to wash your hands regularly.

22 November 2017

Louis-Philippe V ronneau: DebConf Videoteam sprint report - day 3

Erf, I'm tired and it is late so this report will be short and won't include dank memes or funny cat pictures. Come back tomorrow for that. tumbleweed Stefano worked all day long on the metadata project and on YouTube uploads. I think the DebConf7 videos have just finished being uploaded, check them out! RattusRattus Apart from the wonderful lasagna he baked for us, Andy continued working on the scraping scheme, helping tumbleweed. nattie Nattie has been with us for a few days now, but today she did some great QA work on our metadata scraping of the video archive. ivodd More tests, more bugs! Ivo worked quite a bit on the Opsis board today and it seems everything is ready for the mini-conf. \0/ olasd Nicolas built the streaming network today and wrote some Ansible roles to manage TLS cert creation through Let's Encrypt. He also talked with DSA some more about our long term requirements. wouter I forgot to mention it yesterday because he could not come to Cambridge, but Wouter has been sprinting remotely, working on the reviewing system. Everything with regards to reviewing should be in place for the mini-conf. He also generated the intro and outro slides for the videos for us. KiBi and Julien KiBi and Julien arrived late in the evening, but were nonetheless of great assistance. Neither are technically part of the videoteam, but their respective experience with Debian-Installer and general DSA systems helped us a great deal. pollo I'm about 3/4 done documenting our ansible roles. Once I'm done, I'll try to polish some obvious hacks I've seen while documenting.

21 November 2017

Louis-Philippe V ronneau: DebConf Videoteam sprint report - day 2

Another day, another videoteam report! It feels like we did a lot of work today, so let's jump right in: tumbleweed Stefano worked most of the day on the DebConf video archive metadata project. A bunch of videos already have been uploaded to YouTube. Here's some gold you might want to watch. By the end of our sprint, we should have generated metadata for most of our archive and uploaded a bunch of videos to YouTube. Don't worry though, YouTube is only a mirror and we'll keep our current archive as a video master. RattusRattus Andy joined us today! He hacked away with Stefano for most of the day, working on the metadata format for our videos and making schemes for our scraping tools. ivodd Ivo built and tested a good part of our video setup today, fixing bugs left and right in Ansible. We are prepared for the Cambridge Mini-DebConf! olasd Nicolas finished his scripts to automatically spool up and down our streaming mirrors via the DigitalOcean API today and ran our Ansible config against those machines to test our setup. pollo For my part, I completed a huge chunk of my sprint goals: we now have a website documenting our setup! It is currently hosted on Alioth pages, but olasd plans to make a request to DSA to have it hosted on the static.debian.org machine. The final URL will most likely be something like: https://video.debconf.org The documentation is still missing the streaming section (our streaming setup is not final yet, so not point in documenting that) and a section hosting guides for the volunteers. With some luck I might write those later this week. I've now moved on documentation our various Ansible roles. Oh, and we also ate some cheese fondue: Our fondue dinner

20 November 2017

Louis-Philippe V ronneau: DebConf Videoteam sprint report - day 1

Another videoteam report! We've now been hacking for a full day and we are slowly starting to be productive. It's always hard to get back in a project when you haven't touched it in a while... Anyway, let's start this report with some important announcement: we finally have been able to snap a good picture of the airbnb's cat! The airbnb's cat No more nagging me about the placeholder image from Wikipedia I used in yesterday's report! Set up Our hacking space Here's what the team did today: tumbleweed Stefano started the day by hacking away on our video archive. We eventually want to upload all our videos to YouTube to give them exposure, but sadly our archive metadata is in a pretty poor shape. With the script tumbleweed wrote, we can scrape the archive for matches against the old DebConf's pentabarf XML we have. tumbleweed also helped Ivo with the ansible PXE setup he's working on. Some recent contributions from a collaborator implemented new features (like a nice menu to choose from) but also came with a few annoying bugs. ivodd Ivo continued working on the PXE setup today. He also tried to break our ansible setup by using fresh installs with different user cases (locales, interfaces, etc.), with some success. The reason he and Stefano are working so hard on the PXE boot is that we had a discussion about the future of our USB install method. The general consensus on this was although we would not remove it, we would not actively maintain it anymore. PXE is less trouble for multiple machines. For single machines or if you don't control the DHCP server, using ansible manually on a fresh Debian install will be the recommended way. olasd After a very long drive, olasd arrived late in the evening with all our gear. Hurray! We were thus able to set up some test boxes and start wiring the airbnb properly. Tomorrow will certainly be more productive with all this stuff at our disposition. pollo Today I mainly worked on setting up our documentation website. After some debate, we decided that sphinx was the right tool for the job. I am a few pages in and if I work well I think we'll have something to show for at the end of the sprint! I also was thrown back into ansible after witnessing a bug in the locale management. I'm still rusty, but it's slowly coming back to me. Let's end this blog post with a picture of the neon pineapple that sits on the wall of the solarium. Upside down this picture is even more troubling

19 November 2017

Louis-Philippe V ronneau: DebConf Videoteam sprint report - day 0

First day of the videoteam autumn sprint! Well, I say first day, but in reality it's more day 0. Even though most of us have arrived in Cambridge already, we are still missing a few people. Last year we decided to sprint in Paris because most of our video gear is stocked there. This year, we instead chose to sprint a few days before the Cambridge Mini-Debconf to help record the conference afterwards. Since some of us arrived very late and the ones who did arrive early are still mostly jet lagged (that includes me), I'll use this post to introduce the space we'll be working from this week and our general plan for the sprint. House Party After some deliberations, we decided to rent a house for a week in Cambridge: finding a work space to accommodate us and all our gear proved difficult and we decided mixing accommodation and work would be a good idea. I've only been here for a few hours, but I have to say I'm pretty impressed by the airbnb we got. Last time I checked (it seems every time I do, some new room magically appears), I counted 5 bedrooms, 6 beds, 5 toilets and 3 shower rooms. Heck, there's even a solarium and a training room with weights and a punching bag on the first floor. Having a whole house to ourselves also means we have access to a functional kitchen. I'd really like to cook at least a few meals during the week. There's also a cat! Picture of a black cat I took from Wikipedia. It was too dark outside to use mine It's not the house's cat per say, but it's been hanging out around the house for most of the day and makes cute faces trying to convince us to let it come inside. Nice try cat. Nice try. Here are some glamour professional photos of what the place looks like on a perfect summer day, just for the kick of it: The view from the garden The Kitchen One of the multiple bedrooms Of course, reality has trouble matching all the post-processing filters. Plan for the week Now on a more serious note; apart from enjoying the beautiful city of Cambridge, here's what the team plans to do this week: tumbleweed Stefano wants to continue refactoring our ansible setup. A lot of things have been added in the last year, but some of it are hacks we should remove and implement correctly. highvoltage Jonathan won't be able to come to Cambridge, but plans to work remotely, mainly on our desktop/xfce session implementation. Another pile of hacks waiting to be cleaned! ivodd Ivo has been working a lot of the pre-ansible part of our installation and plans to continue working on that. At the moment, creating an installation USB key is pretty complicated and he wants to make that simpler. olasd Nicolas completely reimplemented our streaming setup for DC17 and wants to continue working on that. More specifically, he wants to write scripts to automatically setup and teardown - via API calls - the distributed streaming network we now use. Finding a way to push TLS certificates to those mirrors, adding a live stream viewer on video.debconf.org and adding a viewer to our archive are also things he wants to look at. pollo For my part, I plan to catch up with all the commits in our ansible repository I missed since last year's sprint and work on documentation. It would be very nice if we could have a static website describing our work so that others (at mini-debconfs for examples) could replicate it easily. If I have time, I'll also try to document all the ansible roles we have written. Stay tuned for more daily reports!

4 November 2017

Louis-Philippe V ronneau: Migrating my website to Pelican

After too much time lying to myself, telling myself things like "I'll just add this neat feature I want on my blog next week", I've finally made the big jump, ditched django and migrated my website to Pelican. I'm going to the Cambridge Mini-Debconf at the end of the month for the Debconf Videoteam Autumn sprint and I've taken the task of making daily sprint reports for the team. That in return means I have to publish my blog on Planet Debian. My old website not having feeds made this a little hard and this perfect storm gave me the energy to make the migration happen. Anyway, django was fun. Building a (crappy) custom blogging engine with it taught me some rough basics, but honestly I don't know why I ever thought it was a good idea. Don't get me wrong: django is great and should definitely be used for large and complicated websites. My blog just ain't one. Migrating to Pelican was pretty easy since it also uses Jinja2 templates and generates content from Mardown. The hardest part was actually bending it to replicate the weird and specific behavior I wanted it to have. So yeah, woooo, I migrated to Pelican. Who cares, right? Well, if you are amongst the very, very few people who read the blog posts I mainly write for myself, you'll be please to know that: Here's a bonus picture of a Pelican from Wikimedia, just for the sake of it: A pelican

2 November 2017

Bits from Debian: New Debian Developers and Maintainers (September and October 2017)

The following contributors got their Debian Developer accounts in the last two months: The following contributors were added as Debian Maintainers in the last two months: Congratulations!

3 June 2017

Ingo Juergensmann: Back to the roots: FidoNet - I'm back!

Last month I blogged about Fidonet. This month I can report that I'm back in FidoNet. While I was 2:2449/413 back then, my new node number is now 2:2452/413@fidonet. The old network 2:2449 is still listed in the Fidonet nodelist, but no longer active, but maybe I can revive that network at a later time. Who knows. The other problem I complained last month about was missing software in Debian. There is binkd and ifcico as mailer software and crashmail and ifmail as a tosser, but no reader software. So how did I get started again? First, I got into mood by watching all parts of the BBS documentary about BBSes: It's a nice watch, so even when you don't plan to start a BBS or join Fidonet like I did, you can see Tom Jennings and others talking about BBSes in general and Fidonet. It's somewhat a nice way-back machine and it made me to actually start my comeback to Fidonet. I tried to compile some projects from Sourceforge like fidoip or GoldEdPlus, but all projects were in a state where they didn't compile without additional work under Debian. At least with those included debian/rules that have. So I decided to reactivate my old Fidonet software on my Amiga. Instead of GMS_Mailer I found AmiBinkd on Aminet which runs quite well. With that setup I was able to call to other Fidonet nodes and do some filerequests. That way I found out that 2:2452/250 is one of the still reachable Fidonet boxes in Germany and soon I became 2:2452/413. Still running on my Amiga with Mailmanager as Tosser and Reader and AmiBinkd as a mailer. Using Fidonet is quite different nowadays as you don't need to call out via phone line anymore, but use Internet connections instead. Although this is nice and much faster and with no additional costs and you can use "crash mail", it's not the same fun as dialing into a mailbox by modem and hear the typical sqeaking sound of a modem connecting. So I bought a Zyxel U-1496E modem on Ebay for 5.50 and connected it to my FritzBox 7490. This works quite well and I could place calls via the modem using TrapDoor as a mailer on my Amiga. Anyway, using my Amiga was only a temporary solution to get me up & running again. The goal is to run a full featured Fidonet node on Debian on my colocated server in the datacenter and in the meanwhile I was able to switch the DNS record from my Amiga to the server in the datacenter, running with binkd from Debian and Husky suite as tosser. Husky is complete Fidonet suite, including tosser, areafix, filefix, tic-file processor, etc. However there are no Debian packages available - at least not easily to find. Philipp Giebel pointed me in an Fidonet echoarea to his own personal repository for Debian and Raspbian:
https://www.kuehlbox.wtf/index.php#repo
He was very helpful in getting me started on Linux with Husky and shared many of his config files with me. Big thanks for that! He also used our discussions to write a blog article about this. Although it's German only you can find the necessary config files. You can find that on:
https://www.stimpyrama.org/blog/17-computer/138-ftnsetup
It covers nearly all necessary aspects:
  • how to setup his repo in your apt sources
  • install the necessary packages
  • configuration of husky, binkd and goldedplus with example configs
  • some tips & tricks like some keyboard shortcuts for goldedplus, etc.
So, this is really helpful for everyone that wants to join Fidonet as well. You can use goldedplus as a reader for Fidonet, or when you just want to be a point and not a full node, you might want to try OpenXP on Linux. OpenXP includes everything you'll need for a point, like a mailer, reader and tosser. You can even use it as a mail reader via POP3/IMAP or to read Internet News (aka newsgroups). It's still possible to run a Fidonet node on Amiga, on Linux and of course other operations systems like Windows and even OS/2. And with HotdogEd there is even Fidonet software available on your Android smartphone! But why Fidonet if you already have the Internet at your fingertips? Well, this is something you need to decide for yourself, but for me there are several reasons why I joined Fidonet after 17 years of inactivity again:
  • It's not the Internet! :-) This means basically no spam mails. At least I didn't experience any spam so far.
  • It's a small and welcoming community.
  • There is not only Fidonet itself (with zone 1:* to 5:*), but other zones as well, like for example AmigaNet with zone 39:* or fsxNet with zone 21:*. FTN technology makes it easy to setup a own network based on a certain topic.
  • It's a technology that enabled people to communicate worldwide with each other, long before the Internet was available for everyone! This is some kind of technical heritage I find worthwhile to preserve.
  • Although most people of us can enjoy a free and open Internet, this is not valid for everyone in the world. Nowadays some regimes decide to block and censor the Internet for their citizens. Fidonet or FTN technology can enable those citizens to still communicate free and without censorship when even Tor is not working anymore because the Internet at all has been taken down in a country. Often enough you can still use phone lines and therefor you can use modems to connect to mailboxes and exchange mails and files. FTN is optimized for this kind of dialup connections and this is one of the main reasons why I don't want to only offer connections via Internet but also by modem to my Fidonet node.
So, be invited to join Fidonet as well!
Kategorie:
Debian
Tags:
Debian
Fidonet
Software

4 May 2017

Guido G nther: Debian Fun in April 2017

Debian LTS April marked the 24th month I contributed to Debian LTS under the Freexian umbrella. I had 8 hours allocated plus 4 hours left from March which I used by: Other Debian stuff git-buildpackage Released versions 0.8.14 and 0.8.15. Notable changes besides bug fixes: The versions are also available on pypi.

12 February 2017

Dirk Eddelbuettel: Letting Travis keep a secret

More and more packages, be it for R or another language, are now interfacing different application programming interfaces (API) which are exposed to the web. And many of these may require an API key, or token, or account and password. Which traditionally poses a problem in automated tests such as those running on the popular Travis CI service which integrates so well with GitHub. A case in point is the RPushbullet package where Seth Wenchel and I have been making a few recent changes and additions. And yesterday morning, I finally looked more closely into providing Travis CI with the required API key so that we could in fact run continuous integration with unit tests following each commit. And it turns that it is both easy and quick to do, and yet another great showcase for ad-hoc Docker use. The rest of this post will give a quick minimal run-down, this time using the gtrendsR package by Philippe Massicotte and myself. Start by glancing at the 'encrypting files' HOWTO from Travis itself. We assume you have Docker installed, and a suitable base package. We will need Ruby, so any base Linux image will do. In what follows, I use Ubuntu 14.04 but many other Debian, Ubunti, Fedora, ... flavours could be used provided you know how to pick the relevant packages. What is shown here should work on any recent Debian or Ubuntu flavour 'as is'. We start by firing off the Docker engine in the repo directory for which we want to create an encrypted file. The -v $(pwd):/mnt switch mounts the current directory as /mnt in the Docker instance: 
edd@max:~/git/gtrendsr(master)$ docker run --rm -ti -v $(pwd):/mnt ubuntu:trusty
root@38b478356439:/# apt-get update    ## this takes a minute or two
Ign http://archive.ubuntu.com trusty InRelease
Get:1 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB]
Get:2 http://archive.ubuntu.com trusty-security InRelease [65.9 kB]
# ... a dozen+ lines omitted ...
Get:21 http://archive.ubuntu.com trusty/restricted amd64 Packages [16.0 kB]    
Get:22 http://archive.ubuntu.com trusty/universe amd64 Packages [7589 kB]      
Fetched 22.4 MB in 6min 40s (55.8 kB/s)                                        
Reading package lists... Done
root@38b478356439:/#
We then install what is needed to actually install the travis (Ruby) gem, as well as git which is used by it: 
root@38b478356439:/# apt-get install -y ruby ruby-dev gem build-essential git
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
# ... lot of output ommitted ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
root@38b478356439:/#
This too may take a few minutes, depending on the networking bandwidth and other factors, and should in general succeed without the need for any intervention. Once it has concluded, we can use the now-complete infrastructure to install the travis command-line client: 
root@38b478356439:/# gem install travis
Fetching: multipart-post-2.0.0.gem (100%)
Fetching: faraday-0.11.0.gem (100%)
Fetching: faraday_middleware-0.11.0.1.gem (100%)
Fetching: highline-1.7.8.gem (100%)
Fetching: backports-3.6.8.gem (100%)
Fetching: multi_json-1.12.1.gem (100%
# ... many lines omitted ...
Installing RDoc documentation for websocket-1.2.4...
Installing RDoc documentation for json-2.0.3...
Installing RDoc documentation for pusher-client-0.6.2...
Installing RDoc documentation for travis-1.8.6...
root@38b478356439:/#
This in turn will take a moment. Once done, we can use the travis client to login into GitHub. In my base this requires a password and a two-factor authentication code. Also note that we switch directories first to be in the actual repo we had mounted when launching docker. 
root@38b478356439:/# cd /mnt/    ## change to repo directory
root@38b478356439:/mnt# travis --login
Shell completion not installed. Would you like to install it now?  y  y
We need your GitHub login to identify you.
This information will not be sent to Travis CI, only to api.github.com.
The password will not be displayed.
Try running with --github-token or --auto if you don't want to enter your password anyway.
Username: eddelbuettel
Password for eddelbuettel: ****************
Two-factor authentication code for eddelbuettel: xxxxxx
Successfully logged in as eddelbuettel!
root@38b478356439:/mnt#
Now the actual work of encrypting. For this particular package, we need a file .Rprofile containing a short option() segment setting a user-id and password: 
root@38b478356439:/mnt# travis encrypt-file .Rprofile
Detected repository as PMassicotte/gtrendsR, is this correct?  yes  
encrypting .Rprofile for PMassicotte/gtrendsR
storing result as .Rprofile.enc
storing secure env variables for decryption
Please add the following to your build script (before_install stage in your .travis.yml, for instance):
    openssl aes-256-cbc -K $encrypted_988d19a907a0_key -iv $encrypted_988d19a907a0_iv -in .Rprofile.enc -out .Rprofile -d
Pro Tip: You can add it automatically by running with --add.
Make sure to add .Rprofile.enc to the git repository.
Make sure not to add .Rprofile to the git repository.
Commit all changes to your .travis.yml.
root@38b478356439:/mnt#
That's it. Now we just need to follow-through as indicated, committing the .Rprofile.enc file, making sure to not commit its input file .Rprofile, and adding the proper openssl invocation with the keys known only to Travis to the file .travis.yml.

3 February 2017

Pau Garcia i Quiles: Almost at FOSDEM. Video volunteers?

I am boarding my flight to Brussels to attend FOSDEM. The Desktops DevRoom will be a blast again this year. While I have been in charge of it for 6? years already, the last two (since my twins) were born I had organized remotely and local duties were carried on by the Desktops DevRoom team (thank you Christophe Fergeau, Philippe Caseiro and others!). I am anxious at meeting old friends again. I will be at the beer event today. Video streaming will be available thanks to the Video Team. If you want to help, please contact us in the desktops-devroom@lists.fosdem.org mailing list, or directly at the devroom. Also, this year will be the first for me using the job corner to recruit: my company (everis) is recruiting globally for many open positions. Drop us a mail at fosdem@everis.com with your CV, desired position and location (we have direct presence in 13 countries and indirect in 40 countries) and I will make sure it reaches the right inbox.

2 January 2017

Shirish Agarwal: India Tourism, E-Visa and Hong Kong

A Safe and Happy New Year to all.
Incredible India from http://incredibleindiacampaign.com

Incredible India from http://incredibleindiacampaign.com

While Debconf India is still a pipe-dream as of now, did see that India has been gradually doing it easier for tourists and casual business visitors to come visit India. This I take as very positive development for India itself. The 1st condition is itself good for anybody visiting India
Eligibility International Travellers whose sole objective of visiting India is recreation , sight-seeing , casual visit to meet friends or relatives, short duration medical treatment or casual business visit.
https://indianvisaonline.gov.in/visa/tvoa.html That this facility is being given to 130 odd countries is better still
Albania, Andorra, Anguilla, Antigua & Barbuda, Argentina, Armenia, Aruba, Australia, Austria, Bahamas, Barbados, Belgium, Belize, Bolivia, Bosnia & Herzegovina, Botswana, Brazil, Brunei, Bulgaria, Cambodia, Canada, Cape Verde, Cayman Island, Chile, China, China- SAR Hong-Kong, China- SAR Macau, Colombia, Comoros, Cook Islands, Costa Rica, Cote d lvoire, Croatia, Cuba, Czech Republic, Denmark, Djibouti, Dominica, Dominican Republic, East Timor, Ecuador, El Salvador, Eritrea, Estonia, Fiji, Finland, France, Gabon, Gambia, Georgia, Germany, Ghana, Greece, Grenada, Guatemala, Guinea, Guyana, Haiti, Honduras, Hungary, Iceland, Indonesia, Ireland, Israel, Jamaica, Japan, Jordan, Kenya, Kiribati, Laos, Latvia, Lesotho, Liberia, Liechtenstein, Lithuania, Luxembourg, Madagascar, Malawi, Malaysia, Malta, Marshall Islands, Mauritius, Mexico, Micronesia, Moldova, Monaco, Mongolia, Montenegro, Montserrat, Mozambique, Myanmar, Namibia, Nauru, Netherlands, New Zealand, Nicaragua, Niue Island, Norway, Oman, Palau, Palestine, Panama, Papua New Guinea, Paraguay, Peru, Philippines, Poland, Portugal, Republic of Korea, Republic of Macedonia, Romania, Russia, Saint Christopher and Nevis, Saint Lucia, Saint Vincent & the Grenadines, Samoa, San Marino, Senegal, Serbia, Seychelles, Singapore, Slovakia, Slovenia, Solomon Islands, South Africa, Spain, Sri Lanka, Suriname, Swaziland, Sweden, Switzerland, Taiwan, Tajikistan, Tanzania, Thailand, Tonga, Trinidad & Tobago, Turks & Caicos Island, Tuvalu, UAE, Ukraine, United Kingdom, Uruguay, USA, Vanuatu, Vatican City-Holy See, Venezuela, Vietnam, Zambia and Zimbabwe.
This should make it somewhat easier for any Indian organizer as well as any participants from any of the member countries shared. There is possibility that this list would even get longer, provided we are able to scale our airports and all and any necessary infrastructure that would be needed for International Visitors to have a good experience. What has been particularly interesting is to know which ports of call are being used by International Visitors as well as overall growth rate
The Percentage share of Foreign Tourist Arrivals (FTAs) in India during November, 2016 among the top 15 source countries was highest from USA (15.53%) followed by UK (11.21%), Bangladesh (10.72%), Canada (4.66%), Russian Fed (4.53%), Australia (4.04%), Malaysia (3.65%), Germany (3.53%), China (3.14%), France (2.88%), Sri Lanka (2.49%), Japan (2.49%), Singapore (2.16%), Nepal (1.46%) and Thailand (1.37%).
And port of call
The Percentage share of Foreign Tourist Arrivals (FTAs) in India during November 2016 among the top 15 ports was highest at Delhi Airport (32.71%) followed by Mumbai Airport (18.51%), Chennai Airport (6.83%), Bengaluru Airport (5.89%), Haridaspur Land check post (5.87%), Goa Airport (5.63%), Kolkata Airport (3.90%), Cochin Airport (3.29%), Hyderabad Airport (3.14%), Ahmadabad Airport (2.76%), Trivandrum Airport (1.54%), Trichy Airport (1.53%), Gede Rail (1.16%), Amritsar Airport (1.15%), and Ghojadanga land check post (0.82%) .
The Ghojadanga land check post seems to be between West Bengal, India and Bangladesh. Gede Railway Station is also in West Bengal as well. So all and any overlanders could take any of those ways.Even Hardispur Land Check post comes in the Bengal-Bangladesh border only. In the airports, Delhi Airport seems to be attracting lot more business than the Mumbai Airport. Part of the reason I *think* is the direct link of Delhi Airport to NDLS via the Delhi Airport Express Line . The same when it will happen in Mumbai should be a game-changer for city too. Now if you are wondering why I have been suddenly talking about visas and airports in India, it came because Hong Kong is going to Withdraw Visa Free Entry Facility For Indians. Although, as rightly pointed out in the article doesn t make sense from economic POV and seems to be somewhat politically motivated. Not that I or anybody else can do anything about that. Seeing that, I thought it was a good opportunity to see how good/Bad our Government is and it seems to be on the right path. Although the hawks (Intelligence and Counter-Terrorist Agencies) will probably become a bit more paranoid , their work becomes tougher.
Filed under: Miscellenous Tagged: #Airport Metro Line 3, #CSIA, #Incredible India, #India, #International Tourism

29 December 2016

Philipp Kern: Automating the installation of Debian on z/VM instances

I got tired of manually fetching installation images via either FTP or by manually transferring files to z/VM to test s390x installs. Hence it was about time to automate it. Originally I wanted to instrument an installation via vmcp from another instance on the same host but I figured that I cannot really rely on a secondary instance when I need it and went the s3270/x3270-script way instead.

The resulting script isn't something I'm particularly proud of, especially as it misses error handling that really should be there. But this is not expect instead you operate on whole screens of data and z/VM is not particularly helpful in telling you that you just completed your logon either. Anyway, it seems to work for me. It downloads the most recent stable or daily image if they are not present yet, uploads them via DFT to CMS and makes sure that the installation does not terminate when the script disconnects. Sadly DFT is pretty slow, so I'm stuck with 70 kB/s and about five minutes of waiting until kernel and initrd are finally uploaded. Given that installations themselves are usually blazingly fast on System z, I'm not too annoyed by that, though.

I previously wrote about a parmfile example that sets enough options to bring debian-installer to the point of a working network console via SSH without further prompting. It's a little unfortunate that s390-netdevice needs to be preseeded with the hardware addresses of the network card in all cases, even if only one is available. I should go and fix that. For now this means that the parmfile will be dependent on the actual VM system definition. With that in mind there is an example script in the same gist that writes out a parmfile and then calls the reinstall script mentioned above. Given that debian-installer now supports HTTPS (so far only in the daily images) you can even do a reasonably secure bootstrapping of the network console credentials and preseeding settings.

If you put this pretty generic preseed configuration file onto a securely accessible webserver and reference it from the parmfile, you can also skip the more tedious questions at the beginning of debian-installer. A secure transport is encouraged as preseed files can do anything to your installation process. Unfortunately it seems that there is no way to preseed SSH keys for the resulting installation yet, neither for the created user nor for root. So I haven't achieved my desired target of a fully automated installation just yet. Debian's Jenkins setup just went with insecure defaults, but given that my sponsored VMs are necessarily connected to the public Internet that seemed like a bad idea to me. I suppose one way out would be to IP/password ACL the preseed file. Another one to somehow get SSH key support into user-setup.

Next.

Previous.